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A Topological  Foundation  for  Self -Organization 
Romuald  L Sclbor-Marchockl 


It  is  shown  that  by  the  use  of  Information  Theory,  any  metrizable  topology  may  be  metrized 
as  an  orthogonal  Euclidean  space  (with  a random  Gaussian  probability  distribution)  times  a de- 
numerable random  cartesian  product  of  irreducible  (wrt  direct  product)  denumerable  groups. 

The  necessary  algorithm  to  accomplish  this  metrization  from  a statistical  basis  is  presented,  ff 
such  a basis  is  unavailable,  a certain  nilpotent  projection  operator  has  to  be  used  instead,  as  is 
shown  in  detail  in  the  companion  psy>er.  This  operator  possesses  self -organizing  features. 
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INTRODUCTION 


In  the  companion  article^  we  will  define  a self -organizing  system  as  one  which,  after 
observing  the  Input  and  output  of  an  unknown  phenomenon  (transfer  relation) , organizes  Itself 
Into  a simulation  of  the  unknown  phenomenon. 

Within  the  mathematical  model,  the  aforementioned  phenomenon  may  be  represented  as 
a topological  space  thus  omitting  for  the  moment  the  (arbitrary)  designation  of  Input  and  output 
which,  as  will  be  shown,  bears  on  the  question  of  uniqueness.  Hence,  for  the  purpose  of  this 
paper  which  emphasizes  the  mathematical  foundation,  an  Intelligent  device  is  taken  as  one  which 
carries  out  the  task  of  studying  a space  and  describing  it. 

In  keeping  with  the  policy  that  one  should  not  ask  someone  (or  something)  else  to  do  a task 
that  he  could  not  do  himself  (at  least  In  principle),  let  us  consider  how  we  would  approach  such 
a problem . 

In  the  first  place,  we  have  to  select  the  space  In  which  the  problem  Is  to  be  set.  The  most 
general  space  that  we  feel  capable  of  tackling  is  a metrizable  topology.  On  the  other  hand,  any- 
thing less  general  would  be  unnecessarily  restrictive.  Thus,  we  choose  a metrizable  topological 
space. 

As  soon  as  we  have  made  this  choice,  we  regret  It.  In  order  to  Improve  the  situation 
somewhat,  we  show  that  there  Is  no  (additional)  loss  of  generality  in  using  an  orthogonal  Euclidean 
space  tlmes^  a denumerable  random  cartesian  product  of  Irreducible  (wrt  direct  product)  de- 
numerable groups. 

This  paper  provides  a survey  of  the  problem  and  a method  for  solving  It  which  Is  con- 
ceptually clear  but  not  very  practical.  The  companion  paper^  provides  a practical  method  for 
solving  this  problem  by  means  of  the  successive  use  of  a certain  nllpotent  projection  operator. 


METRIZATION 

We  start  with  a metrizable  topoloelcal  space.  There  are  many  equivalent  axlomatlzatlons 
of  a metrizable  topology;  e^,  see  Kelley  Perhaps  the  easiest  way  to  visualize  a metrizable 
topology  is  to  consider  that  one  was  given  a metric  space  but  that  he  lost  his  notes  in  which  the 
exact  form  of  the  metric  was  written  down.  Thus  one  knows  that  he  can  do  everytlng  that  he  could 
in  a metric  space,  if  only  he  can  figure  out  how. 

The  "figuring  out  how"  is  by  no  means  trivial.  Here,  it  will  be  assumed  that  a cumulative 
probability  distribution  has  been  obtained  on  the  space  by  one  of  the  standard  methods;  bird  In 
cage^,  Munroe^  I,  Munroe®  n,  ordering  (see  Halmos®  or  Kelley®),  or  Kelley®.  This  cumulative 
probability  distribution  is  a function  on  X onto  the  interval  Qo,  1^  of  real  numbers.  The  In- 
verse of  this  function,  which  exists  by  the  Radon  Nlkodym  theorem,  provides  a mapping  from  the 
real  Interval  onto  the  non-trlvlal  portion  of  X.  This  nuq>ping  Induces  all  of  the  pleasant  proper- 
ties of  the  real  numbers  on  the  space  X:  topological,  metric,  and  ordering. 

Actually,  it  turns  out  that,  especially  If  the  dimensionality  of  the  space  Is  greater  than 
one,  the  foregoing  procedure  not  only  provides  one  metrization,  but  nuuiy.  Indeed,  this  lack  of 
uniqueness  is  what  makes  the  procedure  exceedingly  difficult.  Only  by  imposing  some  additional 
conditions  that  result  in  the  existence  of  a unique  solution,  does  the  problem  become  tractable. 
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We  choose  to  Impose  the  additional  condition  that  the  resulting  metric  space  be  a Euclidean 
geometry  with  a rectangular  coordinate  system.  Even  this  always  does  not  yield  uniqueness,  but 
we  will  show  the  additional  restriction  that  will  guarantee  uniqueness  after  the  necessary  language 
is  developed.  Since  all  metrizations  of  a given  metrizable  topology  are  isomorphic,  in  the  quo- 
tient class  the  orthogonal  Euclidean  geometry  serves  the  purpose  of  being  a convenient  repre- 
sentative of  the  unique  element  resulting  from  a given  metrizable  topology. 

Furthermore,  the  same  comment  applies  to  the  use  of  a Gaussian  distribution  as  the  proba- 
bility distribution  on  this  orthogonal  Euclidean  geometry.  Namely,  the  random  Gaussian  distribu- 
tion on  an  orthogonal  Euclidean  geometry  is  a convenient  representative  member  of  the  equivalence 
class  which  maps  into  one  element  (stochastic  space)  of  the  quotient  class. 

Information  Theory 

Now,  we  will  show  that  Information  Theory  provides  the  language  necessary  to  describe 
the  metrizatlon  procedure  in  detail. 

It  is  possible  to  introduce  Information  Theory  axlomatically  by  a suitable  generalization  of 
7 8 

the  axioms  in  Feinstein  . But  to  simplify  the  discussion  here,  we  will  use  the  less  elegant  but 
equivalent  method  of  defining  certain  definite  integrals.  The  probability  density  distribution  p is 
defined  from  the  cumulative  probability  distribution  P by 

(1)  P (X')  = / p(x)  dx  , 

X'  ^ X 

measurable^ 

Then  the  information  rate  H is  defined  as 

> 

(2)  H(X)  = - / p(x)  In  K p(x)  dx  , 

X 

where  kappa  has  (carries)  the  units  of  X . Finally,  the  channel  rate  R is  defined  as 

(3)  R(OXi)  = E H(Xj)  - H(X) 

I I 

o 

is  the  denumerable  cartesian  product  space 

= (S)  Xj 

I 


where  X 
(4)  X 
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Next,  we  define  the  angle  t 


(5)  ie(0  Xj)l  = sln"^  e *i) 

and  the  norm 

(6)  IXI  = « (2»e)‘^/^  e"^^ 

Now,  a statistically  Independent  basis;  one  for  which 

(7)  R(0  Xj)  JL  constant  , 

I 

can  be  provided  in  terms  of  one  dimensional  components;  te,  none  of  them  can  be  decomposed 
further,  then  it  is  just  the  usual  problem  of  diagonalization  of  a symmetric  matrix  by  means  of 
a congruence  transformation  to  provide  an  orthogonal  coordinate  system.  Furthermore,  for 
uniqueness,  we  arrange  the  spectrum  in  decreasing  order.  Then,  by  means  of  the  Radon 
Nikodym  theorem  applied  to  each  of  these  one-dimensional  axes,  the  probability  distribution  may 
be  made;  eg,  Gaussian,  if  desired.  Thus,  we  obtain  the  promised  orthogonal  Euclidean  space. 

Channel 

At  this  time  we  can  state  the  remaining  additional  condition  required  that  a decomposition 
be  unique.  The  index  space  I has  to  be  partitioned  into  exactly  two  parts,  say  I*  and  F'; 

(8)  r U I"  = 1 
r n F’  = 0 


such  that 

(9)  dim  (X')  > dim  (X")  , 

where 

(10)  X'  . © X^ 

r 

X"  - ® X. 

I" 

(If  dim(X)  is  odd,  then  we  have  to  cheat  a little  by  putting  in  an  extra  random  dummy  dimension.) 
And  then  the  decomposition  of  the  space 

(11)  X . ® Xj 


has  to  be  carried  ovt  so  that  this  partitioning  is  preserved.  Since  this  partitioning  is  arbitrary 
(as  far  as  the  mathematics  is  concerned),  it  is  obvious  that  a space  which  is  not  partitioned  will 
have  many  (equivalent)  decompositions.  On  the  other  hand,  if  the  partitioning  is  into  more  than 
two  parts,  then  the  existence  of  a decomposition  is  not  guaranteed. 

A slight  penalty  has  to  be  paid  for  the  use  of  this  partitioning,  namely:  instead  of  eventu- 
ally obtaining  a random  cartesian  product  of  one -dimensional  spaces,  we  obtain  an  extended 
channel  (with  random  input)  of  single -dimensional  channels.  It  is  obvious  that  if  we  were  to  drop 
the  partitioning  temporarily,  each  such  single -dimensional  channel  would  be  further  decomposed 
into  two  random  components.  This  decomposition  is  not  unique.  But  one  of  these  equivalent  de- 
compositions is  particularly  convenient;  namely,  that  decomposition  where  we  take  the  component 
out  of  the  original  X'  and  that  which  is  random  to  it,  say  V . This  V (as  well  as  the  cartesian 
product  of  all  such  Vs,  which  of  necessity  are  random)  is  called  the  linearly  additive  noise. 

The  name  "linearly  additive"  is  justified  because  it  is  just  the  statistical  concept  isomorphic  to 
the  linear  addition  of  vectors  in  orthogonal  Euclidean  geometry.  (The  proof  of  this  last  statement 
is  not  completed  as  yet.) 


Denumerable  Space 

The  procedure  for  this  decomposition  was  worded  to  de-emphasize  the  possible  presence 
of  a denumerable  (component  of  the)  space.  Such  a component  may  be  given  outright;  otherwise, 
it  results  iff  the  space  was  not  simply  connected.  Any  denumerable  space  is  zero  dimensional, 
as  may  be  verified  easily  from  the  full  information  theoretic  definition  of  dimensionality. 

The  obvious  way  of  disposing  of  a denumerable  space  is  to  use  the  conventional  mapping^ 
that  converts  a Stieltjes  to  a Lebeaque  Integral,  using  fixed  length  segments.  (It  can  be  shown 
that  H is  invariant  under  such  a mapping. ) Unfortunately,  while  this  mapping  followed  by  a 
repetition  of  the  preceding  procedure  will  always  solve  a given  problem  (no  new^^  denumerable 
component  need  be  generated  on  the  second  pass),  little  insight  is  provided  into  the  structure  of 
the  resulting  space.  On  the  other  hand,  because  channels  under  cascading  constitute  a group, 
any  such  denumerable  space  is  a representation  of  a denumerable  group. 


SUMMARY 

In  summary,  the  orginal  metrizable  topological  space  was  decomposed  into  an  orthogonal 
Euclidean  space  tlmes^  a denumerable  random  cartesian  product  of  irreducible  (wrt  direct 
product)  denumerable  groups.  Thus,  since  any  individual  component  of  a random  caiiesian 
product  may  be  studied  Independently  of  the  others,  all  that  one  needs  to  study  is;  (1)  a Gaussian 
distribution  on  a single  real  axis  and  (2)  the  irreducible  denumerable  groups. 

Finally,  it  should  be  emphasized  that  there  are  only  these  two  ways  of  decomposing  a 
metrizable  topology;  (1)  if  a (statistical)  basis  is  given,  use  the  diagonalization  of  a symmetric 
matrix  algorithm  described  earlier  (and  given  in  detail  in  the  three  channels  in  cascade  problem),  and 
(2)  otherwise  use  a suitable  network  of  the  NPO's  with  n^  * 1.  Of  course,  any  hybrid  of  these 
two  methods  may  be  employed  as  well. 
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If  I is  infinite,  certain  precautions  have  to  be  exercised. 

^^This  "if"  is  the  catch  that  makes  ^ methods  of  metrtzatton  of  a space  of  dimensionality 
higher  than  one  impractical,  except  the  method  of  successive  projections  upon  unit  spheres  cen- 
tered at  the  center  of  gravity.  The  method  of  using  that  nilpotent  projection  operator  is  described 
in  the  companion  paper 

^^Only  non-cyclic  irreducible  (wrt  direct  product)  denumerable  group  components  of  the 
old  denumerable  space  will  remain. 
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